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NAVTRAEQUIPCEN  IH-300 
SUMMARY  AND  CONCLUSIONS 

This  report  documents  a one-year  effort  in  which  some  of  the  results  of 
an  earlier  research  and  development  project  were  prepared  for  implementation 
at  a Recruit  Training  Command.  This  project  concerned  computer-based  in- 
struction for  recruit  company  commanders  in  the  affective  and  communication 
skills  areas.  In  an  ongoing  continuation  of  this  effort,  the  more  complete 
preparation,  implementation,  and  evaluation  of  these  capabilities  are  under- 
way. 


The  programs  involved  the  use  of  a stand-alone  computer-based  system  for 
evaluating  and  training  company  commanders  and  for  obtaining  information 
needed  for  managing  Recruit  Training  Command  operations  in  general.  In  ac- 
complishing these  functions,  the  computer  controlled  the  operation  of  a video 
tape  player  and  a card  reader  in  addition  to  some  more  traditional  components 
i of  a computer-based  training  system.  A new  computer  language  was  developed 

especially  for  facilitating  the  transfer  of  some  programs  previously  prepared 
in  PLATO  IV  coding  to  a form  compatible  with  the  current  computer  system. 

Preliminary  efforts  to  evaluate  the  feasibility  and  desirability  of  these 
programs  indicated  that  the  system  can  operate  as  intended  in  the  operational 
setting  and  that  its  acceptance  by  Recruit  Training  Comaand  personnel  is 
generally  favorable.  The  results  also  indicated  that  the  systan  would  entail 
relatively  small  expenditures  to  acquire,  operate,  and  maintain;  especially 
when  one  considers  the  higher  costs  and  lower  quality  to  be  expected  if  these 
training  and  management  functions  were  performed  by  other  means.  The  ultimate 
desirability  of  performing  these  functions  by  any  means  is  a more  complex 
issue,  as  it  is  for  almost  any  of  the  operations  of  large  organizations  such 
as  a Recruit  Training  Command.  The  criticality  of  this  issue,  however,  demands 
that  greater  attention  be  given  to  Its  resolution. 

The  current  effort  is  considered  only  an  introductory  demonstration  of  the 
types  of  tasks  that  can  be  profitably  assigned  to  a computer  at  Recruit 
Training  Commands.  The  cost  effectiveness  of  the  systan  would  be  expected  to 
increase  markedly  as  other  applications  are  Included  along  with  the  present 
one.  This  report  suggests  a variety  of  projects  for  future  research  and 
development  that  would  appear  to  contribute  to  the  cost-effectiveness  of  a 
system  similar  to  the  present  one. 


i 


1 


NAVTRAEQUIPCEN  IH-300 
PREFACE 


| This  project  was  jointly  funded  by  the  Navy  Personnel  Research  and  Develop- 

■ ment  Center  and  the  Office  of  Naval  Research.  (The  Office  of  Naval  Research 

i supported  the  effort  related  to  the  video  tape  portions  of  the  training  mate- 

; rials.)  This  report  documents  the  development  of  a computerized  evaluation 

and  training  system  (CETS)  for  use  at  the  Recruit  Training  Conmands  (RTCs); 
as  well  as  the  initial  phases  of  implementing  and  evaluating  this  system. 

The  original  plan  called  for  a brief  (two-month)  field  evaluation  of  the  sys- 
| tem.  This  has  been  expanded  to  extend  the  evaluation  period  throughout  FY  78. 

The  evaluation  will  include  assessments  of  the  operation,  acceptance,  usage, 
and  effects  of  the  system  at  RTC,  Orlando.  Some  of  the  implementation  activ- 
ities will  be  conducted  by  RTC,  Orlando  with  support  and  evaluation  assistance 
provided  by  NAVTRAEQUIPCEN. 

J Many  people  at  RTC,  Orlando  were  very  helpful  in  this  effort.  Most  no-  ' 

ij  tably,  LCDR  William  Sullivan,  NCI  Ginger  Simpson,  CTRCS  Zetterholm,  all  the 

[i  division  officers  who  helped  in  the  development  and  promotion  of  the  system, 

ij  all  the  company  commanders  who  participated  in  the  video  taping  and  the 

i company  conmander  students. 

j 

j The  complex  and  significant  tasks  of  computer  systems  programming  were 

j accomplished  by  Mr.  Marty  Smith.  The  innovative  computer  programming 

approaches  for  the  video  tape  developments  were  accomplished  by  Mr.  Pat  Smith. 

Mr.  Fred  Qelmos  assisted  in  the  preparation  of  video  tape  training  materials 
and  he  and  Mr.  Pat  Smith  were  responsible  for  day-to-day  operation  of  the 
system. 
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NAVTRAEQUIPCEN  IH-300 
SECTION  I 
INTRODUCTION 

It  is  generally  recognized  that  interpersonal  communications  and  relation- 
ships are  critically  important  to  organizational  operations.  Still,  relative- 
ly little  assistance  seems  to  be  provided  toward  improving  the  performance  of 
these  activities.  In  an  effort  to  ameliorate  this  situation,  the  possibility 
that  computer  technology  might  be  beneficially  applied  to  this  area  has  been 
investigated. 

Thus,  the  NAVTRAEQUIPCEN  has  been  engaged  over  the  past  five  years  in 
determining  a number  of  alternative  approaches  to  the  use  of  computers  for 
improving  interactions  among  people  and  in  testing  and  evaluating  several  of 
these  approaches  in  operational  settings.  The  operational  settings  for  these 
efforts  have  been  Recruit  Training  Conroands  (RTCs),  and  the  focus  of  the 
efforts  within  the  RTCs  has  been  primarily  on  training  and  evaluating  Inter- 
personal aspects  in  the  role  of  the  recruit  company  commander  (CC).  The 

12  3 4 5 

earlier  programs  and  evaluations  are  described  in  other  reports.  ’ * ’ ’ 

The  current  project  was  directed  toward  preparing  the  latest  version  of 
these  training  programs,  demonstrated  in  the  first  half  of  1976,  for  adoption 
by  the  RTC,  Orlando.  This  version  of  the  training  consisted  of  a collection 
of  instructional  approaches  and  materials  that,  after  many  Iterations,  were 
designed  to  meet  the  research  and  operational  requirements  and  conditions  that 
prevailed  at  that  point  in  time.  Thus,  due  to  time  and  funding  restrictions, 
many  potentially  viable  components  of  instruction  developed  in  previous 
phases  of  this  project  were  not  seriously  considered  for  inclusion. 

The  current  phase  of  this  project  continued  the  effort  to  comply  with  re- 
search and  operational  requirements.  However,  this  phase  represents  the  most 
concerted  action  to  date  on  this  project  to  prepare  a training  package  for 
acceptance  and  adoption  by  the  RTC  as  a standard  component  of  their  operation. 

The  endeavor  to  meet  RTC  requirements  Involved  updating,  modifying,  and 
deleting  existing  training  programs  and  developing  new  training  and  evaluation 


T Spencer,  G.  j,  and  Hausser,  D.  1.;  Blalwes,  A,  S.  and  Weller,  D.  R.  Use 
of  Computer-Assisted  Instruction  for  Interpersonal  Skill  Training  - A Pilot 
Study.  1975.  Technical  Report:  NAVTRAEQUIPCEN  73-C -0133-1. 

2.  Cohen,  0.  1.  and  Flshbein,  M.  Development  and  Research  Utilizing  the 
PLATO  IV  System  for  Company  Conrmmder  Behavioral  Change  Training.  August 

1975.  Technical  Report:  NAVTRAEQUIPCEN  73-C-0129-1. 

3.  Hausser,  D.  L.,  Blalwes,  A.  S.,  Weller,  D.  R.,  and  Spencer,  G.  J.  Applica- 
tion of  Computer-Assisted  Instruction  to  Interpersonal  Skill  Training.  January 

1976.  Technical  Report:  NAVTRAEQUIPCEN  74-C-0100-1. 

4.  Cohen,  J.  L.  and  Ftshbeln,  M.  A Field  Test  of  the  PLATO  IV  System  for 
Company  Commander  Behavioral  Change  Training.  July  1976.  Technical  Report: 
NAVTRAEQUIPCEN  74-C-0095-1. 

5.  Lukas,  G.,  Blalwes,  A.  S.,  Weller,  D.  R.  Evaluation  of  Human  Relations 
Training  Programs.  January  1977.  Technical  Report:  NAVTRAEQUIPCEN 
75-C-0076-1. 
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programs.  In  addition,  it  involved  the  selection  and  procurement  of  an  appro- 
priate new  computer  system  which  had  capabilities  similar  to  but  which  was 

more  economical  than  the  PLATO  IV  system®  used  previously. 

This  report  discusses  the  procedures  and  results  related  to  this  effort  to 
update  and  extend  the  earlier  versions  of  the  training  and  to  implement  and 
evaluate  the  new  computerized  evaluation  and  training  system  (CETS)  that  re- 
sulted. 


Welter,  David  V.  Using  PLATO  IV.  CERL,  University  of  Illinois,  Urbana, 
October  1975. 


6 


NAVTRAEQUIPCEN  IH-300 
SECTION  II 
S . STEM  DEVELOPMENT 

It  was  jointly  determined  by  NAVTRAEQUIPCEN  and  RTC,  Orlando  that  the 
computer-based  training  system  should  include  two  major  components:  (a) 
Evaluation  - a new  evaluation  program  for  assessing  the  background,  perform- 
ance, and  perceptions  of  CCs  and  recruits,  and  the  relationship  of  these 
variables  to  the  operation  of  the  organization;  and  (b)  Training  - a training 
program,  made  up  of  both  existing  and  new  training  materials,  which  would  pro- 
vide both  computer-based  instruction  and  recommendations  to  correct,  by  other 
means  (e.g.,  counseling),  weaknesses  revealed  by  the  evaluation  program. 

NAVTRAEQUIPCEN  then  acquired  a computer  system  capable  of  handling  these 
two  components,  with  cost-effectiveness  the  primary  consideration.  Necessary 
software  systems  and  evaluation  and  training  materials  were  developed  and 
implemented. 

Procedures  for  designing  and  developing  the  CETS  system  are  described  in 
the  following: 

EVALUATION  PROGRAM 

The  evaluation  program  was  originally  intended  to  provide  information  on 
the  performance  of  CC's  and  to  suggest  remedial  action  where  needed.  However, 
it  soon  became  apparent  that  many  other  uses  could  be  made  of  the  system,  due 
to  the  nature  and  amount  of  data  being  collected  and  the  processing  power  of 
the  computer.  Therefore,  the  evaluation  program  evolved  into  a general  manage- 
ment tool  for  RTC  in  addition  to  its  evaluation  role  for  CCs. 

DATA  con  ECTION.  Data  for  the  evaluation  program  were  collected  from  a variety 
of  sources,  as  seen  in  Table  1.  All  data  were  collected  near  the  time  of 
graduation  of  each  recruit  training  unit  at  RTC.  The  Recruit  Survey,  Division 
Officer  (DO)  Evaluation,  and  CC  Survey  were  developed  jointly  by  RTC  and 
NAVTRAEQUIPCEN  personnel. 

In  the  selection  of  items  for  these  questionnaires,  a wide  range  of  rele- 
vant areas  was  considered.  A list  of  tentative  items  was  drawn  from  several 
sources  Including  the  scientific  literature  on  leadership  and  performance 
evaluation,  the  leadership  and  evaluation  practices  of  the  Navy  (especially 
those  of  RTC),  and  results  from  previous  phases  of  the  project.  The  item  lists 
were  organized  into  categories  by  sorting  the  items  into  homogeneous  groups 
and  then  defining  and  titling  each  group.  Items  were  deleted  and  added,  and 
groups  were  redefined  to  assure  maximum  coverage  and  clarity  with  minimum 
redundancy.  In  addition,  00s,  division  adjutants  and  CC  school  personnel 
assessed  the  items  and  categories  for  their  appropriateness,  which  provided 
tlie  bases  for  additional  modifications.  The  list  was  divided  into  three  survey 
forms  according  to  who  was  best  qualified,  due  to  his  position  at  RTC,  to 
answer  each  question  --  recruits,  DOs,  or  CCs.  The  three  survey  forms  appear 
in  Appendix  A (category  Information  has  been  added  to  these  forms  for  purposes 
of  this  report). 
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The  purpose  of  the  Recruit  Survey  was  to  measure  the  CC's  behavior  in  the 
areas  of  instruction  and  interpersonal  activities,  and  to  measure  the  morale 
and  attitudes  of  recruits. 

The  purpose  of  the  Division  Officer  Evaluation  was  to  measure  the  CC's 
performance  on  the  job  and  the  training  unit's  quality. 

The  purpose  of  the  CC  Survey  was  to  provide  measures  of  the  CC's  opinions 
and  attitudes  about  various  aspects  of  RTC  and  his  job. 

The  forms  were  all  answered  on  optical  scanning  computer  cards,  and  input 
to  the  system  for  storage  and  data  manipulation  through  a card  reader. 

TA81E  1 . DATA  COLLECTION  FOR  EVALUATION  PROGRAM 


OATA SOURCE  NO.  OF  ITEMS 


1. 

CC  Behaviors 

Recruit  Survey  (filled 
out  by  all  recruits  of 
a given  CC) 

36 

2. 

CC  Performance 

Division  Officer  Evalua-  ' 
tion  (filled  out  by  the 
Division  Officer  of  a 
given  CC) 

31 

3. 

CC  Perceptions  and 
Background 

CC  Survey  (filled  out  by 
the  Company  Cnnmander) 

37 

4. 

Recruit  Attitudes 
(morale) 

Recruit  Survey 

11 

5. 

RTC  System  Oata 
(Includes  T.U.  back- 
ground and  perform- 
ance, and  some  CC 
background  items) 

RTC  Records 

16 

OATA  PROCESSING  ANO  DISPLAYS. 

The  operation  of  the  evaluation  program  is 

diagrammed  in  Figure  1.  Two  basic  types  of  data  output  available  were 
grouped  data  and  individual  data.  The  grouped  data  option  gave  the  average 
score  of  a particular  group  of  CCs  (e.g.,  all  CCs  in  Division  3)  on  a selected 
variable,  while  the  Individual  option  gave  the  score  of  a single  CC  on  a 
selected  variable. 

Displays  of  data  were  available  at  three  levels  of  specificity: 

a.  Item  data  Involved  scores  on  single  items; 

b.  Cateqory  data  Involved  scores  on  groups  of  related  items  (e.g., 
all  items  on  the  Division  Officer  Evaluation  related  to  "Bearing"); 
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Figure  1.  Evaluation  Program  Flow  Diagram 
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c.  Summary  data  involved  total  scores  from  a given  source  (e.g.,  total 
score  from  Division  Officer  Evaluation). 

As  depicted  in  Figure  1,  the  amount  of  data  displayed  depended  on  the 
security  class  of  the  user.  The  Commanding  Officer  (and  other  authorized 
personnel)  could  see  all  system  data.  DOs  could  see  all  data  on  CCs  in  their 
divisions.  CCs  could  see  all  the  data  which  pertained  to  them.  A password 
security  system  prevented  any  user  from  seeing  data  which  was  not  within  his 
security  class. 

The  Systems  Monitoring  component  provided  the  information  by  which  the 
system  operator  determined  which  data  needed  to  be  entered  into  the  computer 
and  the  means  by  which  the  data  was  entered. 

The  outputs  were  presented  on  the  cathode-ray  tube  (CRT)  screen,  and, 
optionally,  in  hard  copy  form.  All  outputs  were  tailored  specifically  to  the 
users'  needs.  To  this  end,  RTC  personnel  were  consulted  frequently  during  the 
design  of  the  outputs  to  ensure  that  they  were  easily  understandable,  and  in 
a form  most  suitable  for  their  intended  use. 

An  example  of  an  individual  CC  data  output  is  provided  in  Appendix  B.  It 
shows  the  scores  of  a given  CC  on  a group  of  category  variables,  as  well  as 
the  average  scores  of  all  CCs  in  the  division,  all  CCs  of  the  same  rank,  all 
CCs  in  the  training  group,  and  all  CCs  at  RTC,  In  addition,  percentile  scores 
are  given  to  show  how  the  CC  compares  with  other  CCs  in  his  division,  other 
CCs  of  the  same  rank,  and  all  CCs  at  RTC. 

This  type  of  output  was  used  to  point  out  the  strengths  and  weaknesses  of 
CCs,  so  that  they  might  take  corrective  action  on  the  weak  areas.  The  weak 
and  strong  areas  are  easily  determined  from  the  percentile  scores. 

This  output  was  used  by  00s  for  several  purposes  including:  (a)  assisting 
the  DO  in  counseling  CCs;  (b)  providing  inputs  to  official  evaluations;  (c) 
informing  the  00  of  the  performance  of  his  division;  (d)  helping  to  determine 
areas  of  strength  and  weakness  in  the  division;  (e)  evaluating  the  effects  of 
any  actions  taken  by  the  00. 

This  output  can  be  used  by  the  Comuanding  Officer  and  other  appropriate 
personnel  as  a management  tool,  for  use  any  time  detailed  data  (both  absolute 
and  relative  scores)  concerning  any  of  the  variables  on  the  system  is 
required. 

An  example  of  a grouped-CC  data  output  is  shown  in  Appendix  B,  where  the 
average  scores  on  several  category  variables  are  given,  grouped  by  division. 

It  Is  also  possible  to  sort  tlx;  data  by  training  group,  training  unit,  CC 
rank,  and  CC  sex.  This  type  of  output,  available  to  the  Commanding  Ofiicer 
and  other  authorized  personnel,  provides  a management  aid  which  is  useful  any 
time  it  is  desired  to  compare  the  perfornunce  of  various  groups  of  CCs. 

In  summary,  the  evaluation  program  is  designed  to  give  information  to  the 
Commanding  Officer,  DOs,  CCs,  and  other  personnel  (e.g.,  Code  40  staff,  CC 
school  personnel)  on  the  performance  of  CCs  and  RTC  in  general,  and  to  eval- 
uate the  effects  of  any  changes  in  RTC  operations. 


10 


NAVTRAEQU IPCEN  IH-300 


The  evaluation  program  has  some  similarity  to  the  RATES  (Rapid  Access 

Training  Evaluation  System^)  program  once  used  at  RTC.  The  RATES  system  also 
used  a computer  to  process  attitudinal  data.  RATES,  however,  utilized  only 
fairly  raw  measures  of  student  attitude  as  evaluation  criteria,  whereas  CETS 
goes  beyond  this  to  provide  highly-processed  attitudinal  data  from  a variety 
of  sources.  In  addition,  CETS  relates  a wealth  of  potential  evaluation 
criteria,  also  in  a highiy-interpretable  form,  from  a variety  of  other  sources 
(e.g.,  the  organizational  performance  measures,  the  performance  ratings,  the 
demographic  data).  Thus,  the  CETS  evaluation  program  is  similar  to  RATES  in 
that  CETS  includes  many  of  RATES  features;  however,  CETS  Is  broader  than  RATES 
in  both  the  extensiveness  of  its  capabilities  and  the  range  of  its  application. 

CONTRIBUTIONS  OF  THE  COMPUTER.  A computer  is  essential  to  the  operation  of  an 
evaluation  system  such  as  this  since  it  can:  (a)  input  and  stor'  vary  large 
quantities  of  Information;  (b)  rapidly  perform  complex  calculatii.iS  and 
manipulations  on  this  information;  and  (c)  quickly  provide  displays  and  out- 
puts which  present  the  processed  information  in  the  most  usable  form. 

TRAINING  PROGRAM 

LESSONS  PRESENTED  ON  COMPUTER  TERMINAL.  During  previous  phases  of  this  proj- 
ect, training  materials  utilizing  computer-assisted  instruction  were  developed 

n 

under  four  approaches.  These  were:  (1)  the  skill  training  approach,  (2)  the 
9 10 

behavior  training  approach,  (3)  the  evaluation  approach,  and  (4)  the  case 

study  approach.'1  Of  these,  only  the  first  was  considered  acceptable  for  oper- 
ational use  at  RTC  at  this  time.  The  decision  to  exclude  the  other  three 
approaches  was  based  on  several  considerations.  The  skill  training  program 
was  the  most  complete,  having  been  under  development  throughout  the  history 
of  the  project.  It  was  also  the  approach  which  had  the  most  substantial 
and  favorable  evaluation  results.  Resource  limitations  did  not  permit  the 
Implementation  of  the  other  three  approaches  although  they  still  have  good 
promise  for  application,  given  the  time  and  resources  necessary  to  develop 
them  further. 

Some  modifications  were  made  1 r.  the  skill  training  programs  as  they  were 
transferred  to  CETS.  These  modifications  were  primarily  in  two  areas.  The 
programs  were  made  applicable  to  female  CCs  (they  had  been  developed  for 
males),  and  the  new  RTC  terminology  and  practices  were  incorporated  (e.g., 
"Battalion  Commander"  became  "Division  Officer"). 


T.  Hums h Resource  No rva gemcn t Project  Final  Report.  June  30,  1976,  S.F.  and  G. 
dba  Mercury,  San  Oiego,  Calif,  Contract  N-001 23-C-01 91  (unpublished  report). 

8.  See  footnotes  1 and  3 on  page  a. 

9.  See  footnotes  2 and  4 on  page  S, 

10.  See  footnotes  2 and  4 on  page  5. 

11.  See  footnote  5 on  page  5. 
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The  operation  of  this  portion  of  the  training  program  is  shown  in  Figure 
2 (blocks  1 through  10),  After  receiving  an  introduction  to  the  system,  the 
student  was  given  instruction  on  the  "Three  Keys"  (Goal  Setting,  Instruction, 
and  Feedback),  followed  by  lessons  on  each  of  the  "Seven  Skills."  A portion 
of  the  material  from  the  lesson  "Concrete"  is  included  in  Appendix  C.  These 
materials  are  more  fully  described  in  previous  reports  (see  footnotes  1,  3 and 
5). 

Contributions  of  the  Computer.  The  contributions  of  a computer  to  a training 
program  such  as  this  include  all  of  those  usually  associated  with  computer- 
assisted  instruction,  viz.,  improved  self-pacing  of  students  and  individuali- 
zation of  instruction;  and  automatic  remediation,  testing,  data  keeping,  etc. 

In  addition,  computer-assisted  instruction  provides  an  added  benefit  in  the 
area  of  interpersonal  skills  training,  in  that  the  absence  of  human  evaluators 
during  training  provides  a relatively  nonthreatening  atmosphere.  This  is  not 
true  of  some  other  methods  of  instruction  in  this  area,  such  as  role-playing. 

LESSONS  UTILIZING  VIDEO  TAPE.  The  new  training  material  developed  during  this 
phase  involved  computer-controlled  video-tape  presentations  of  CCs  and  re- 
cruits. These  lessons  were  intended  to  provide  more  real-to-life  representa- 
tions of  the  CC's  job  for  training  and  testing  purposes.  Video  tape  was  ex- 
pected to  be  an  efficient  means  of  increasing  student  interest  in  the  training 
as  well  as  transfer  from  training  to  the  job  Itself. 

These  benefits  were  expected  to  result  from  an  increase  in  the  student's 
belief  that  the  skills  being  taught  actually  are  effective  with  recruits.  The 
credibility  of  the  instruction  should  increase  because  the  student  can  see  the 
program  skills  being  performed  well  and  poorly  by  actual  CCs  in  job  situations 
wherein  the  apparent  effects  on  recruits  correspond  with  the  quality  of  per- 
formance (i.e.,  the  recruits  respond  well  to  the  good  performance  and  less  well 
to  the  poor  performance).  The  video  tapes  also  were  expected  to  help  reduce 
ambiguity  In  the  Instructional  concepts  by  providing  concrete  examples  of  their 
Implementation. 

Shadowing.  An  alternate  method  of  achieving  these  benefits  Is  practiced  at 
RTCs  in  the  form  of  "shadowing."  Here,  the  student  observes  one  or  more  CCs 
performing  normal  job  activities.  Whereas  shadowing  Is  Important  for  getting 
a general  Impression  of  the  Job,  the  video  tape  lessons  supplement  the 
shadowing  experience  in  the  following  ways: 

a.  Variety  - The  student  can  be  shown  a greater  number  and  variety  of 
situations  and  approaches  for  dealing  with  them  on  video  tape  than  would  be 
practical  with  shadowing; 

b.  Interpretation  - The  lossons-to-be-learned  are  indicated  in  the  video 
tape  programs,  whereas  in  shadowing  they  are  generally  left  to  the  student's 
self-instructional  abilities; 

c.  Permanency  - Video  tape  lessons  can  be  used  long  after  the  CCs  ap- 
pearing in  them  have  left  RTC.  A CC  can  only  be  shadowed,  of  course,  while  he 
remains  at  RTC; 
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Figure  2.  Training  Program  Flow  Diagram 


d.  Standardization  - With  video  tape,  each  student  Is  assured  of  experi- 
encing the  subject  matter  considered  most  critical,  whereas  In  shadowing  the 
criticality  of  the  subject  matter  is  left  to  chance; 

e.  Review  - Students  can  see  critical  video  tape  Instruction  again  and 
again,  whereas  In  shadowing  the  lesson  is  often  gone  before  the  student  is 
ready  to  start  learning; 

f.  improvement  - With  the  video  tape  programs,  the  Instruction  can  con- 
tinually be  updated  and  Improved,  whereas  the  Instructional  quality  In  shadow 
ing  varies  fairly  randomly  over  time,  depending  on  the  quality  of  the  CCs 
being  shadowed; 

g.  Efficiency  - The  video  tape  lessons  "compress"  the  shadowing  experi- 
ence by  editing  out  nonsignificant  events  and  idle  time,  thus  saving  student 
time. 

Content  of  Video  Tapes.  As  shown  In  Figure  2,  there  were  eight  video  tape 
lessons.  Each  lesson  Involved  several  segments  of  video  tape  training 
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material  (from  three  to  eight).  The  duration  of  the  segments  varied  from  a 
few  seconds  to  over  a minute.  The  content  of  the  tapes  was  varied  by  showing 
several  different  CCs  in  a number  of  different  situations.  These  situations 
involved  conrnon  activities  such  as:  instruction  (on  marching,  academic 
subjects,  locker  stowage,  etc.);  holding  individual  and  group  counseling  and 
feedback  sessions;  anu  distributing  proficiency  awards. 

Further  variation  in  the  content  of  the  video  tape  materials  was  achieved 
by  addressing  both  positive  and  negative  aspects  of  a CC's  performance.  For 
example,  two  vignettes  were  enacted  to  emphasize  two  general  types  of  commonly 
observed  inadequacies  among  CCs.  One  type  of  general  inadequacy  was  called 
"wishy  washy"  because  the  CC  was  very  attentive  to  the  emotions  and  morale  of 
the  recruit,  but  not  very  directive.  The  other  enacted  vignette  showed  a 
"tough  guy"  type  of  deficiency  which  was  just  the  reverse  of  "wishy  washy" 

(very  directive,  but  ignoring  emotional  aspects).  All  of  the  other  vignettes 
were  abstracted  from  video  tapes  depicting  naturally  occurring,  on-the-job 
encounters  between  CCs  and  recruits.  CCs  were  selected  for  taping  by  the  RTC 
if  their  schedule  included  important  interactions  with  recruits  during  the 
taping  period.  After  some  Initial  nervousness,  most  CCs  appeared  to  feel 
fairly  comfortable  in  the  taping  situation.  Some  video  tapes  were  taken  from 
about  30  yards  away  from  the  subjects  (with  the  aid  of  a wireless  microphone 
and  te’ephoto  lens)  which  seemed  to  help  decrease  the  obtrusiveness  of  the 
video  taping  activities  and  thereby  the  discomfort  of  the  participants. 

Operation  of  Video  Tape  Lessons.  As  shown  in  Figure  2,  the  video  tape 
materials  were  presented  to  the  students  as  the  final  portion  of  the  training 
program.  The  textual  portion  of  each  lesson  appeared  on  the  graphics  terminal , 
while  the  video  portion  appeared  on  an  adjacent  TV  monitor. 

The  computer  automatically  controlled  all  functions  of  the  tape  player, 
freeing  the  student  to  attend  to  the  training  material.  Introductory  material 
for  each  tape  segment  appeared  on  the  graphics  terminal.  This  was  followed  by 
presentation  of  the  tape  segment  on  the  video  monitor  which,  in  turn  was 
followed  by  questions  on  the  terminal.  After  each  segment,  the  student  had 
the  option  of  answering  the  questions  or  re-viewing  the  tape  segment  first. 

In  some  cases,  a wrong  answer  to  a question  would  result  in  the  tape  segment 
being  reshown  automatically.  In  many  Instances,  the  questions  following  a 
segment  would  progress  from  very  general  to  quite  specific;  first  asking  the 
student  to  identify  general  skill  areas  being  displayed,  then  asking  for  iden- 
tification of  specific  behaviors  which  occurred  within  the  skill  areas. 

Contributions  of  the  Computer.  This  marriage  between  the  computer  and  the 
video  tape  represents  an  advancement  to  the  state-of-the-art  in  using  video 
tapes  for  Instruction  in  the  interpersonal  skills  area,  as  well  as  for  In- 
struction in  general. 

Video  taped  presentations  of  job  performance  have  been  successfully  util- 
ized for  training  purposes  for  many  years.  The  application  of  video  tape 
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techniques  to  interpersonal  skill  instruction  has  included  areas  such  as  guid- 
ance and  counseling,'2’'2’'4  Psychiatric  therapy,'5  classroom  instruction,'5 
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group  problem  solving,  and  job  interview  training. 

The  instructional  strategies  employed  in  these  efforts  have  ranged  from 
merely  showing  a continuous  video  tape  to  the  student  with  little  or  no 
orienting  instructions  to  interactive  sessions  wherein  an  instructor  interprets 
and  discusses  the  action  on  the  video  tape  with  the  student.  The  interactive 
strategies  provide  greater  assurance  that  the  student  understands  the  lessons 
contained  in  the  video  tape.  However,  this  approach  is  limited  by  the  ability 
of  the  instructor,  and  it  requires  a considerable  investment  of  instructor 
time.  The  present  application  has  the  advantages  of  the  interactive  strategies 
without  the  limitations  in  that  the  computer  directs  the  student's  attention 
to  critical  aspects  of  the  video  tape  and  elicits,  evaluates,  and  corrects  his 
reactions  to  these  stimuli.  A further  advantage  of  a computer- control  led  video 
tape  system  is  that  it  can  locate  the  desired  sections  of  tape  much  more  ac- 
curately and  rapidly  than  a human  operator,  thus  avoiding  undesirable  breaks 
in  the  continuity  of  the  lessons. 

COMPUTER  SYSTEM 

HARDWARE.  The  training  programs  described  in'  the  previous  section  were  pre- 
viously implemented  on  the  PLATO  IV  computer  system.  It  was  decided  to  trans- 
fer this  training  material  to  a new  stand-alone,  mini-computer  based  system. 

The  requirements  for  this  new  system  were  developed  with  the  following  factors 
in  mind: 


a.  Ability  to  execute  all  previously  developed  training  material . 


Kagan,  N.  Can  technology  Help  Us  Toward  Reliability  in  Influencing  Human 
Interaction?  Educational  Technology,  February  1973,  44-51. 

13.  Tarrier,  Rl  Hew  Trends  In  Technology  Management  for  Training  Group 
Counselors.  Educational  Technology,  February  1973,  52-56. 

14.  Cohn,  8.  Absentee-Cueing : A Technical  Innovation  in  the  Training  of 
Group  Counselors.  Educational  Technology,  February  1973,  61-62. 

15.  Onder,  J.  The  Use  of  Television  in  Psychiatric  Treatment  and  Education. 
Presented  at  Society  of  Motion  Picture  and  Television  Engineers  103rd  Technical 
Conference,  May  5-10,  1968, 

16.  Rutherford,  R.  The  Effects  of  a Model  Video  tape  and  Feedback  Video  Tapes 

on  the  Teaching  Styles  of  Teachers  in  Traininq.  Journal  of  Experimental  Educa- 
tion, Vol . 42.  No.  1,  Fall,  1973,  64-69.  — 

IT.  Walter,  G.  Acted  Versus  Natural  Models  for  Performance-Oriented  Behavior 
Chanqe  in  Task  Groups.  Journal  of  Applied  Psychology,  1975.  Vol.  60.  No.  3. 
303-307 . 

18.  Venardos,  M.  G.  and  Harris,  M,  B.  Job  Interview  Training  with  Rehabili- 
tation Clients:  A Comparison  of  Videotape  and  Role  Playing  Procedures. 

Journal  of  Applied  Psychology.  1973,  Vol.  58,  No.  3,  365-367. 
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b.  Ability  to  execute  all  proposed  training  material;  especially  the  CC 
evaluation  program  and  the  video  tape  training  materials. 

c.  Cost. 

d.  Delivery  schedule. 

e.  System  Software  capabilities. 

f.  Programming  languages  available. 

g.  Compatibility  with  other  NAVTRAEQUIPCEN  computer  equipment. 

h.  Maintenance  availability. 

i.  Reliability. 

j.  Availabll Ity  on  GSA  schedule. 

k.  Expandability. 

The  system  selected  was  centered  around  a Data  General  NOVA-3  mini-com- 
puter. The  primary  factors  influencing  the  choice  were  delivery  schedule  and 
compatibility  with  other  NAVTRAEQUIPCEN  equipment.  The  presence  of  other  NOVA 
computers  at  NAVTRAEQUIPCEN  made  possible  the  programming  of  much  of  the 
material  prior  to  the  delivery  of  the  actual  system,  since  transfer  of  pro- 
grams between  NOVA  computers  is  relatively  easy. 

Principal  components  of  the  system  are: 

a.  NOVA  3/12  CPU  with  32K  of  MOS  memory. 

b.  Data  General  10  mega -byte  cartridge  disc  system. 

c.  Oata  General  60  cps  printer/tcrminal . 

d.  Tektronix  CRT  graphics  terminal. 

e.  Oata  General  mark  sense  optical  card  reader. 

f.  Sony  V0-2600  video-tape  recorder  with  Interface  to  NOVA. 

The  hardware  system  configuration  is  shown  graphically  in  Figure  3. 

The  total  cost  of  this  system  was  approximately  $35,000,  and  maintenance 
costs  are  approximately  $5,000  per  year. 

SOFTWARE.  The  software  for  the  CETS  system  can  be  categorized  into  three 
areas:  (1)  lessons  presented  on  the  terminal;  (2)  lessons  using  video  tape; 
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and  (3)  the  evaluation  program.  A detailed  description  of  the  software  can 
19 

be  found  in  another  report. 

Lessons  Presented  on  the  Terminal.  These  lessons  had  previously  been  imple- 
mented on  the  PLATO  IV  system.  It  was  necessary  to  translate  them  to  a form 
compatible  with  the  new  system.  For  this  purpose,  a major  effort  was  invested 
in  developing  a new  programming  language  for  the  new  system  very  similar  to  the 
TUTOR  language  used  on  PLATO  IV.  This  made  the  translation  process  relatively 
easy  in  that  much  of  the  old  TUTOR  code  could  be  typed  directly  into  the  new 
system.  Also,  some  of  the  important  characteristics  of  the  TUTOR  language  were 
retained,  the  primary  one  being  programming  simplicity.  It  is  relatively  easy 
for  a non-programmer  to  learn  the  language  in  a short  time. 

After  a program  was  written  in  this  new  language,  it  was  then  put  through  a 
translation  program.  This  program  translated  the  code  from  the  new  language 
into  Fortran  and  Assembler  routines  which  the  computer  then  compiled  and  exe- 
cuted. With  minor  exceptions,  the  programs  operated  as  they  did  on  the  PLATO 
IV  system. 

Lessons  Using  Video  Tape.  These  lessons  were  also  writt0"  in  the  new  language. 
However,  they  included  special  commands  for  the  operation  of  the  video  tape 
player.  The  computer  was  able  to  control  ail  functions  of  the  tape  player 
(play,  record,  fast  forward,  rewind,  stop,  pause)  through  an  interface  which 
also  was  designed  especially  for  use  in  this  project.  Timing  was  used  to  lo- 
cate specific  points  on  the  tape.  For  example,  if  the  next  segment  to  be 
played  was  farther  along  on  the  tape,  the  computer  would  activate  "fast 
forward"  for  the  required  number  of  seconds,  then  "stop";  then  "play"  for  the 
required  number  of  seconds.  To  find  a segment  earlier  in  the  tape,  "rewind" 
would  be  activated  for  the  appropriate  time. 

Evaluation  Program.  Software  *or  the  evaluation  program  consisted  of  programs 
to  input,  store,  manipulate,  and  display  the  large  amounts  of  data  which  were 
involved  in  the  system.  Dataware  input  through  the  terminal  keyboard  and  the 
card  reader.  All  data  were  sorted  by  training  unit  number  and  stored  on  disk. 
Statistics  were  calculated  on  all  data,  including  means,  standard  deviations, 
z-scores,  and  percentile  scores.  These  statistics  were  used  in  the  creation 
of  the  displays,  as  shown  in  Appendix  B.  As  shown  in  these  displays,  it  was 
possible  to  group  the  data  in  many  ways  (by  rank,  by  division,  by  training 
group,  etc,).  Fach  of  these  displays  could  be  output  on  either  the  CRT  termi- 
nal or  the  printer. 


TT,  Appli-Mation  tnc.  Program  Description  Document  for  the  Computerized 
Evaluation  and  Training  System.  October  1977. 
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SYSTEM  IMPLEMENTATION  AND  EVALUATION 

In  addition  to  assuring  that  an  appropriate  evaluation  and  training  system 
is  developed,  it  is  critical  to  the  success  of  the  project  that  the  new  pro- 
grams be  properly  integrated  into  the  ongoing  operations  at  RTC;  that  they  be 
appropriately  utilized  by  RTC  personnel;  and  that  an  assessment  be  made  of  the 
contribution  of  this  program  to  the  RTC  operation. 

INTEGRATION  INTO  RTC 

The  integration  of  CETS  into  RTC  was  facilitated  by  maintaining  a contin- 
ual liaison  between  the  users  at  RTC  and  the  Research  and  Development  (R&D) 
team  at  NAVTRAEQU  IPCEN  during  the  developmental  period  and  by  commencing  the 
transfer  of  control  of  the  new  training  to  RTC  personnel.  The  RTC's  ability 
to  operate  the  system  is  being  facilitated  by  a system  operator's  manual  which 
outlines  operational  procedures  and  by  the  computer  programs  which  were  de- 
signed to  facilitate  the  interaction  between  the  human  users  and  the  computer- 
based  system.  The  computer  programs  assist  the  Integration  process  in  that 
most  aspects  of  the  training  and  evaluation  programs  operate  automatically, 
thus  eliminating  the  need  for  major  deviations  from  ongoing  routines  of  RTC 
personnel.  The  operation  of  the  system  requires  a monitor  to  be  available  to 
assist  in  case  of  an  occasional  system  malfunction,  but  routine  operating 
procedures  require  only  about  10  man-hours  per  week.  It  should  be  noted  that 
these  10  man-hours  are  not  all  added  effort,  in  that  these  routine  procedures 
include  several  tasks  which  were  performed  by  RTC  before  the  Introduction 
of  CETS. 

The  transfer  In  system  operations  has  been  occurring  over  a six-month 
period  which  initiated  with  demonstrations  of  the  system  and  is  continuing 
with  the  actual  operation  of  aspects  of  the  system  by  RTC  personnel  with 
NAVTRAEQU IPCEN  support  as  required. 

Some  problems  were  encountered  in  connection  with  this  integration 
process.  Many  of  these  problems  are  attributable  to  the  rapid  transition  of 
CETS  from  research  and  development  stages  to  an  operational  system.  For 
example,  hardware  and  software  malfunctions  which  were  due  mainly  to  the 
early  development  stage  of  the  system  caused  some  delays  in  the  training  of 
CCs  and  in  the  processing  of  needed  data,  and  this  created  problems  for  the 
RTC  staff.  However,  the  system  has  become  more  stable  over  time,  and  such 
problems  are  becoming  more  Infrequent. 

Another  area  of  difficulty  involved  the  acceptance  of  CETS  by  RTC  person- 
nel. It  took  time  for  word  of  its  purposes  and  capabilities  to  spread 
throughout  RTC.  Since  it  is  a radically  new  means  of  training  and  evaluating 
CCs,  many  mistaken  ideas  about  the  system  arose,  especially  among  CCs  then>- 
selves.  These  mistaken  ideas  were  combated  with  numerous  briefings  and 
memoranda  from  NAVTRAEQUIPCEN  and  RTC  personnel.  The  use  of  the  system  by 
CCs  has  increased  from  almost  nil  at  the  beginning  of  the  project  to  about 
50%  of  available  CCs  at  the  present  time,  and  It  shows  promise  of  continued 
Improvement.  Although  progress  toward  CC  acceptance  appears  to  be  advancing 
well,  this  area  will  require  constant  vigilance  and  nurturing,  especially 
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until  the  system  loses  its  experimental  status  and  becomes  an  integral  part 
of  RTC. 

Acceptance  of  CETS  by  other  RTC  personnel  was  predicated  on  somewhat  dif- 
ferent considerations  than  for  CCs.  One  area  of  concern  for  these  personnel 
was  the  lack  of  concrete  evidence  that  the  current  configuration  of  programs, 
in  the  form  of  CETS,  was  having  a positive  effect  on  the  organization.  The 
provision  of  such  evidence  is  a time-consuming  process.  It  involves  training 
or  evaluating  a large  number  of  CCs,  waiting  until  they  have  picked  up  and  led 
subsequent  companies,  and  then  evaluating  them  again.  Following  this,  the 
data  must  be  analyzed  to  determine  where  changes  in  performance  have  occurred. 
This  process  takes  several  months,  and  the  first  complete  data  are  just  new 
being  collected.  Some  of  the  necessary  evidence  concerning  the  effects  of 
CETS  will  soon  be  available. 

UTILIZATION  AT  RTC 

The  training  portion  of  CETS  was  used  by  RTC  on  a routine  scheduled  basis 
to  train  prospective  CCs,  i.e.,  those  who  had  just  completed  the  standard  CC 
school  (a  four-week  program  of  lecture-oriented  classes).  Following  the  com- 
puter-assisted training,  the  CC  students  entered  a period  of  shadowing.  After 
a few  weeks  of  shadowing,  they  were  given  their  first  training  unit  (T.U.)  to 
lead.  This  involved  preparing  70-80  recruits,  in  an  eight -week  period,  to  be 
capable  of  assuming  life  in  the  Navy. 

The  evaluation  portion  of  the  system  came  into  play  at  the  end  of  the 
training  period,  when  all  Inputs  (Table  1)  were  collected  and  processed.  Each 
CC  was  encouraged  to  come  to  the  computer  and  look  at  his/her  data,  in  order 
to  become  aware  of  his/her  weak  and  strong  points.  RTC  and  NAV T1LAEQU I PCEN 
personnel  were  present  to  instruct  CCs  in  use  of  the  evaluation  system,  and  to 
suggest  steps  which  the  CCs  might  take  to  improve  their  performance. 

In  addition  to  the  mainly  scheduled  utilization  by  new  CCs,  the  system  was 
also  used  by  other  RTC  personnel  in  a variety  of  ways.  For  example: 

a.  Experienced  CCs  looked  at  data  from  their  training  units  to  determine 
their  strong  and  weak  points. 

b.  Oata  from  t : system  have  been  used  by  CC  school  personnel  to  assist  in 
the  counsel ing  of  CCs. 

c.  The  Director  of  the  Standards  and  Evaluations  Department  has  made  ex- 
tensive use  of  data  outputs  to  investigate  questions  such  as: 

(1)  how  do  variables  such  as  age,  rank,  rate,  etc.  relate  to  CC  per- 
formance? 

(2)  what  variables  determine  a CC 1 s attitude  toward  CC  school? 

(3)  what  variables  are  related  to  the  differences  in  the  performance 
of  the  various  divisions? 

d.  The  CETS  data  has  been  used  to  help  select  the  "Sailor  of  the  Month." 
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ASSESSMENT  AT  RTC 

The  CETS  system  was  evaluated,  in  a preliminary  analysis,  from  several 
points  of  view  during  the  period  of  transfer  of  control  from  NAVTRAEQUIPCEN 
to  RTC.  Results  from  this  evaluation  are  categorized  as  follows: 

a.  Attitudes  of  the  RTC  staff  (CCs,  DOs,  CC  school  personnel,  and  others 
who  used  the  system)  toward  the  system. 

b.  Amounts  of  use  made  of  the  various  components  of  the  system. 

c.  Cost  savings  resulting  from  use  of  the  system. 

ATTITUDES  OF  RTC  STAFF.  The  majority  of  the  attitude  data  came  from  student 
CCs  who  went  through  the  training  portion  of  the  system.  A total  of  18  stu- 
dent CCs  have  completed  the  training  to  date.  Each  student  was  given  two 
attitude  forms  during  training;  one  after  the  training  materials  which  ap- 
peared on  the  terminal,  and  one  after  the  video  tape  training.  The  questions 
and  results  are  shown  in  Table  2. 

Table  2 shows  that,  on  Form  I,  the  students  considered  the  instruction  to 
be  interesting  (#1 ) and  useful  to  them  (#2).  There  was  a tendency  to  say 
that,  as  a result  of  the  instruction,  they  were  more  able  (#3)  and  more 
likely  (M)  to  perform  the  skills  that  were  taught.  They  also  thought  that 
RTC  would  be  improved  if  more  of  its  personnel  used  the  skills  (#5). 

Form  II  shows  that  the  video-tape  materials  improved  the  responses  on  four 
of  the  five  Items,  although  none  of  the  differences  was  statistically  signifi- 
cant. Also,  the  results  on  items  6 and  7 show  that  the  video  tape  programs 
Increased  the  enjoyment  and  perceived  effectiveness  of  the  training  beyond 
that  of  the  printed  material  alone.  Across  both  forms,  76*  of  the  responses 
were  favorable,  37  unfavorable,  and  21*  neutral. 

Much  Improved  picture  and  sound  quality  are  possible  in  the  video  tape 
materials,  and  some  criticism  from  students  and  other  RTC  personnel  was 
directed  at  these  elements.  It  is  encouraging  to  note,  however,  that  the 
advantages  of  the  video  tape  approach  were  sufficiently  impressive  so  as  to 
elicit  favorable  overall  attitudes  in  spite  of  these  technical  shortcomings. 
Further,  most  of  the  taping  problems  were  due  to  the  inexperience  of  the 
researchers  in  video  taping  techniques.  The  preparation  of  good  quality 
sound  and  video  should  be  much  less  a problem  now  that  such  experience  has 
been  attained. 

No  formal  measures  were  taken  of  attitudes  of  others  on  the  RTC  staff. 
However.  Judging  from  the  extensive  utilization  by  non-CC  personnel  (see  next 
section),  it  can  be  assumed  that  the  system  was  perceived  as  a useful  tool. 

AMOUNT  OF  USE,  The  computer  kept  track  of  the  amount  of  time  that  the  system 
was  usea  by  various  categories  of  personnel.  Also,  detailed  use  data  was 
kept  on  the  student  CCs.  They  were  scheduled  to  use  the  system  for  two  hours 
per  day  each  until  they  completed  the  training.  They  took  an  average  of  6,01 
hours  to  complete  the  training  (range  4.07  to  7.87). 
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TABLE  2.  CC  STUDENT  ATT ITUDES 


ITEM 


FORM  I FORM  II  DIFFERENCE 
MEAN  MEAN  (II-I) 


1.  The  recent  experience  with  the  CETS  3.94 

program  was:  (5  = fascinating,  3 = neither 

liked  nor  disliked,  1 = a bore) 

2.  The  recent  experience  with  the  CETS  4.12 

program  is:  (5  = very  useful,  3 = neither 

useful  nor  useless,  1 = useless) 

3.  I think  that  I am  more  able  to  perform  3.44 
the  skills  of  the  CETS  program  as  a result 

of  my  recent  experiences  with  the  procram. 

(5  = strongly  agree,  3 = unsure,  1 « strongly 
disagree) 

4.  I think  that  I am  now  more  likely  to  3.50 

actually  use  the  skills  of  the  CETS  program 

as  a result  of  my  recent  experiences  with 
the  program.  (5  = strongly  agree,  3 » 
unsure,  1 = strongly  disagree) 

5.  RTC  would  be  more  effective  if  more  of  3.94 
Its  personnel  would  use  the  skills  taught 

in  the  CETS  training  program.  (5  = strongly 
agree,  3 = unsure,  1 = strongly  disagree) 

6.  Without  the  video-tape  materials,  the  NA* 
CETS  program  would  have  been  less  enjoyable. 

(5  o strongly  agree,  3 ® unsure,  1 = 
strongly  disagree) 

7.  Without  the  video-tape  materials,  the  NA* 
CETS  program  would  have  been  less  effective. 

(5  =>  strongly  agree,  3 » unsure,  1 0 strongly 
disagree) 

* Not  Applicable 


3.88 

4.31 

3.81 

3.75 

4.06 

4.25 

4.19 


-.06 


+.19 


+ .37 


+ .25 


+.12 


NA* 


NA* 


A total  of  36  experienced  CCs  used  the  system  for  an  average  of  22  minutes 
each,  primarily  to  look  at  their  evaluation  data.  Most  of  this  use  occurred 
in  the  last  couple  of  weeks  of  the  three-month  period  of  operational  applica- 
tion. Usage  by  other  RTC  staff  members  is  difficult  to  categorize  accurately. 
This  is  because  a request  for  data  would  often  come  from  one  person,  but 
another  person  would  sign  on  the  computer  to  output  the  cata.  For  example, 
much  of  the  computer  time  charged  to  NAVTRAEQUIPCEN  staff  members  was  actually 
used  to  fill  requests  by  RTC  personnel  for  data. 

However,  an  overall  estimate  of  the  time  that  the  system  was  in  use  by  (or 
for)  RTC  staff  personnel  other  than  CCs  is  130  hours  and  tills  too  is  Increasing 
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at  an  accelerated  rate.  Personnel  using  the  system  included  the  Coimianding 
Officer,  Military  Training  Officer,  Director  of  Standards  and  Evaluations, 
division  officers,  and  CC  school  personnel. 

COST  SAVINGS  RESULTING  FROM  USE  OF  SYSTEM.  The  best  indicators  of  the  cost- 
effectiveness  of  the  CETS  system  appear  to  involve  the  post-RTC  performance  of 
recruits  and  the  speed  of  their  progress  through  RTt.  Except  ^or  attrition 
rates  and  to  some  extent  perhaps  recruit  competition  scores,  such  measures 
generally  are  not  available.  Many  of  the  variables  that  can  be  related  to  the 
ultimate  cost-effectiveness  of  a program  at  RTC  either  are  almost  inextricably 
enmeshed  in  the  complexities  of  the  Navy  system  or  have  a somewhat  question- 
able connection  to  fiscal  considerations. 

Examples  of  some  cost-related  variables  at  RTC  that  are  difficult  to 
measure  due  to  system  complexities  are  the  time  saved  and  the  problems  solved 
due  to  rhe  availability  of  data  from  the  evaluation  programs.  To  determine 
the  nature  of  such  effects,  one  would  need  to  know  what  data  were  used  from 
the  CETS  system,  how  much  resources  would  have  been  required  to  get  these  data 
by  other  means,  what  actions  were  made  possible  by  virtue  of  the  availability 
of  the  data,  and  what  would  have  been  the  consequences  of  not  taking  these 
actions.  It  appears  that  valid  information  of  these  kinds  would  be  difficult 
to  obtain.  However,  an  evaluation  effort  in  this  direction  will  be  made  in 
the  ongoing  evaluation  of  CETS. 

The  attitudes,  morale,  and  motivations  of  RTC  personnel  are  also  difficult 
to  measure  accurately  and  there  is,  additionally,  a less  clear-cut  relation- 
ship between  these  measures  and  cost  considerations.  In  some  of  the  earlier 
referenced  reports  on  this  project  {Spencer,  et.  al.,  1975,  Hausser,  et.  al., 
1976,  Lukas,  et.  al.,  1977),  it  was  noted  that  the  recruits  of  better  CCs 
(i,«.,  those  who  were  rated  higher  by  their  supervisors:  who  had  companies 
with  higher  competition  scores  and  lower  attrition;  and  who  performed  at 
higher  skill  levels  as  judged  by  their  recruits)  were  significantly  more 
likely  to  report  favorable  attitudes,  morale,  and  motivations  than  those  of 
lower-rated  CCs.  These  findings  support  the  notion  that  these  affective 
measures  are  indeed  measures  of  recruit  quality.  Thus,  in  the  absence  of  some 
of  the  mure  direct  cost-effectiveness  measures,  affective  data  can  contribute 
to  the  picture  of  RTC  operations  from  which  cost  effectiveness  can  be  in- 
ferred. Sufficient  affective  data  are  not  yet  available  to  support  an  infer- 
rence  about  the  present  cost  effects  of  CETS.  Based  on  findings  from  the 
earlier  research  on  this  project,  however,  it  is  expected  that  such  effects 
due  to  CETS  will  be  positive. 

Tangible  evidence  of  cost-savings  is  obtainable  where  CETS  is  substituted 
tor  a less  efficient  manner  of  performing  a job.  If  the  tasks  pc . formec  uy 
CETS  were  to  be  attempted  manually,  many  times  the  resources  would  be  required 
to  provide  services  that  would  only  be  poor  approximations  to  the  accuracy, 
comprehensiveness,  and  speed  of  the  CETS  system.  Unfortunately,  however,  most 
of  the  tasks  performed  by  CETS  either  were  not  performed  previously  at  RTC  or 
a substitution  approach  to  its  evaluation  was  not  employed. 

One  area  where  CETS  was  substituted  for  a previous  procedure,  however,  is 
that  of  the  processing  oT^recruit  critique  sheets."  Performed  manually,  this 
operation  required  about  six  hours  a week  to  orocess  about  1/5  the  data  as  is 
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now  being  processed  automatically  by  CE1S.  Thus,  CETS  is  providing  more  sub- 
stantial information  in  seconds  vice  hours.  Continuing  attention  will  be  given 
to  other  opportunities  at  RTC  to  replace  a less  efficient  operation  with  the 
automatic  capabilities  of  CETS. 

A prime  example  of  this  is  the  potential  of  replacing  some  portions  of  the 
regular  CC  training  program.  The  costs  of  the  time  that  instructors  normally 
invest  into  the  teaching  taken  r 'er  by  CETS  could  be  deducted  from  the  costs 
of  CETS,  thusly  further  contributing  to  Its  cost-effectiveness. 

The  pursuit  of  cost-effectiveness  evaluations  typically  is  considered  un- 
wieldly  if  not  impractical  for  areas  such  as  the  present  one  (involving  leader- 
ship, interpersonal  communications,  etc.).  However,  the  criticality  of  the 
findings  and  the  potential  for  contributions  to  a general  methodology  for  con- 
ducting such  investigations  require  continued  efforts  in  this  direction. 
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CURRENT  ACTIVITIES  AND  FUTURE  PROPOSALS 

Ongoing  activities  consist  of  completing  the  development  of  the  evalua- 
tion program;  continuing  the  testing,  refinement,  and  further  development  of 
the  training  programs;  and  initiating  the  evaluation  of  the  total  system. 

EVALUATION  PROGRAM 

Plans  for  FY  78  efforts  at  extending  the  evaluation  program  include: 

a.  Addition  of  a remediation  feature  to  the  CC  outputs  which  will  suggest 
actions  to  be  taken  to  correct  any  deficiencies  uncovered  by  the  evaluation 
(i.e.,  going  through  portions  of  the  skills  training  programs,  counseling  with 
the  DO,  etc.). 

b.  A computer-based  procedure  for  weighting  and  combining  various  evalua- 
tion variables  for  obtaining  global  {and  perhaps  official)  evaluations  of  CC 
performance. 

c.  Revalidation  of  evaluation  variables. 

d.  Statistical  tests  (e.g.,  t-statistics,  correlations,  regression  analy- 
sis, analysis  of  variance)  to  determine  the  statistical  significance  of  rela- 
tionships among  variables. 

e.  Graphical  displays  of  relationships  among  variables. 

f.  Displays  which  compare  existing  groups  (e.g.,  divisions,  ranks,  male/ 
female)  on  their  scores  for  all  variables. 

g.  Automatic  printouts  of  messages  to  personnel  requesting  updating  of 
data  for  the  evaluation  program. 

h.  Scheduling  of  official  MED  inspections. 

TRAINING  PROGRAMS 

Further  development  of  the  training  programs  will  Include: 

a.  Continuing  refinement  of  existing  programs  based  on  data  from  current 
users. 


b.  Improvement  of  sound  quality  and  computer  control  of  existing  video 
tapes. 


c.  Development  of  a program  in  which  the  computer  assists  judges  to  eval- 
uate video  taped  performances  and  to  provide  performance  feedback  to  the  people 
who  are  the  subjects  of  the  video  tapes. 


25 


NAVTRAEQU I PC  EN  IH-300 


SYSTEM  EVALUATION 

Current  procedures  for  evaluating  the  CETS  system  involve  determining  who 
uses  the  various  components  of  the  system,  how  they  use  it,  their  attitudes 
toward  it,  and  the  impact  of  the  system  on  the  behavior  and  performance  of 
recruits  and  CCs,  and  on  organizational  effectiveness. 

FUTURE  PROPOSALS 

Proposals  for  the  further  use  and  expansion  of  the  CETS  system  will  be 
based  on  feasibility  and  cost-effectiveness  estimates  of  such  applications  as: 

a.  Scoring  and  summarizing  recruit  test  performances  and  incorporating 
these  data  into  the  current  evaluation  program  for  further  processing  and 
analysis  (as  is  done  with  the  current  variables). 

b.  Collecting  and  analyzing  data  on  prospective  CC  school  candidates  for 
the  purpose  of  tailoring  CC  training  to  the  individual. 

c.  Collecting  and  analyzing  fleet  data  on  the  performance  of  recruits 
after  they  leave  RTC  as  an  additional  source  of  management  information. 

d.  Computer  administration  of  major  portions  of  the  CC  training  program 
as  currently  conducted  at  RTC, 

e.  Implementing  an  In-depth  program  for  assisting  RTC  personnel  in  the 
proper  Interpretation  and  utilization  of  data  from  the  evaluation  program. 

f.  Collecting  and  analyzing  data  related  to  other  variables  or  positions 
at  RTC,  and  administration  of  similar  training  to  other  positions  at  RTC 
(e.g.,  classroom  instructors). 

g.  Employing  the  video-tape  capabilities  for  providing  feedback  to  CCs 
on  their  own  on-the-job  performances. 

h.  Creating  video-tape  instruction  for  administration  to  large  groups  of 
students. 

1.  Applying  the  computer  system  to  the  management  of  Instruction  for  re- 
cruits (thusly  allowing  each  to  progress  at  his  own  pace  through  some  parts  of 
RTC). 

j.  Giving  recruits  brief  exposure  to  CAI  (to  prepare  them  for  the  type  of 
training  many  of  them  will  receive  after  RTC). 

k.  Performing  many  of  the  training  analysis  functions  of  he  data  analysis 
department  at  RTC. 
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SECTION  V 
DISCUSSION 

This  project  demonstrates  some  rather  novel  capabilities  and  applications 
of  an  instruction-oriented,  computer-based  system  at  an  RTC.  It  also 
demonstrates  some  atypical  approaches  to  training,  evaluation,  and  management 
in  the  interpersonal -conmunication  area.  The  demonstration  is  considered 
introductory  because  the  system  can  do  much  more  at  relatively  little  addition- 
al cost;  thus  increasing  its  cost  effectiveness.  It  is  introductory  also  be- 
cause, as  for  any  new  approach,  continual  evaluation  and  refinement  is  required 
to  assure  maximal  exploitation  of  existing  capabilities.  The  arguments  for 
the  use  of  the  system  as  is  and  for  expanding  its  use  further  are  based  on  the 
benefits  already  achieved  and  on  those  expected  from  future  efforts. 

The  expected  future  benefits  were  addressed  in  Section  IV  of  this  report. 
The  benefits  already  achieved  include  results  obtained  from  the  three  previous 
evaluations  of  materials  related  to  this  project  as  well  as  the  findings  of  the 
current  evaluation.  The  previous  research  (which  was  identical  in  many  aspects 
to  the  present  research)  has  found  the  following  effects  due  to  the  training 
program:  (1)  improved  behaviors  in  CCs;  (2)  improved  attitudes  in  recruits; 

(3)  some  evidence  of  Improved  performance  in  recruits. 

The  current  research  provided  some  critical  data  on  the  operability  and 
acceptability  of  the  present  version  of  the  computer-based  programs  for 
training,  evaluation,  and  management  functions  at  RTC.  First,  the  favorable 
attitudes  of  the  students  (Section  III)  provided  evidence  for  the  acceptabil- 
ity and  face  validity  of  the  Instructional  components.  Second,  the  data  con- 
cerning the  use  of  the  evaluation  program  indicated  that  this  component  of  the 
sjstem  Is  well  on  its  way  to  fulfilling  its  Intended  functions.  This  feature 
was  intended  to  be  used  as  a management  tool  for  diagnosing  problem  areas 
prior  to  action;  as  a basis  for  determining  action;  and  in  evaluating  the 
effects  of  actions  after  they  are  Implemented.  Although  RTC  Is  just  becoming 
familiar  with  this  capability  and  much  of  the  data  base  still  needs  to  be 
acquired  for  some  operations,  the  evaluation  system  has  already  been  used 
extensively  for  a variety  of  purposes  (Section  III). 

In  addition  to  being  effective,  the  programs  are  efficient,  due  mainly  to 
the  computer.  The  computer  plays  a central  and  critical  role  in  all  phases  of 
the  current  system,  as  was  explained  In  Section  II  ("contributions  of  the  com- 
puter"). Some  of  the  expected  cost  benefits  or  those  already  noted  or  infer- 
red have  been  described  in  Section  III.  Thus,  considering  the  benefits  al- 
ready apparent,  the  costs  of  operating  and  maintaining  the  system  (detailed  in 
Section  II)  appear  to  be  justified.  The  justification  for  a system  such  as 
this  becomes  even  more  compelling  when  one  considers  the  potential  benefits  to 
be  derived  from  additional  applications  of  the  system.  These  additional  ap- 
plications would  be  relatively  easy  to  accomplish  now  that  the  basic  system  is 
operational . 
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APPENDIX  A 

DATA  COLLECTION  SYSTEM 


RECRUIT  SURVEY 

The  following  pages  contain  questions  about  you,  your  CC,  and  the  Navy  in 
general.  Answer  each  question  by  making  a mark  on  the  NAVTRAEQUIPCEN  answer 
form  which  you  have  been  given.  Mark  the  form  as  follows: 

A if  you  strongly  agree  with  the  statement. 

B if  you  agree  with  the  statement. 

C if  you  neither  agree  nor  disagree  with  the  statement. 

TT  if  you  disagree  with  the  statement. 

E if  you  strongly  disagree  with  the  statement. 

Example:  If  question  1 said  "Florida  is  hot  in  the  summer"  and  you  strongly 
agree  with  this,  you  would  make  a mark  in  the  A column  after  number 

1. 

A B C D E 

1.  - 

Make  your  marks  with  a black  pencil.  Do  not  put  your  name  on  the  answer 
form,  but  do  fill  in  your  company  number  and  day  of  training.  Your  company 
commander  will  not  know  how  you  answered.  He  or  she  will  only  see  the  average 
answers  given  by  the  entire  company. 


A. 

GENERAL 

B. 

3 KEYS 

C. 

7 SKILLS 

1. 

COUNSELING 

1. 

GOA!.  SETTING 

1. 

CONCRETE 

2. 

MOTIVATING 

2. 

INSTRUCTION 

2. 

TIMELY 

3. 

INSTRUCTING 

3. 

FEEDBACK 

3. 

CLARIFYING 

4. 

RESPECT  FOR  CC 

4. 

REASONABLE 

5. 

GENERAL  MORALE 

5. 

RELEVANT 

6. 

LIKING  FOR  BOOTCAMP 

6. 

CONSIDERATE 

7. 

ESPRIT  DE  CORPS 

7. 

HUMAN 

8. 

INTENTION  TO  RE- 
ENLIST 

9. 

EXPECTED  LIKING  OF 
NAVY 

10. 

VALUE  OF  TRAINING 

11. 

REWARD  & PUNISHMENT 

Figure  A-l,  Recruit  Survey 
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-J 

§ 

1 

5 

E 

STRONGLY 

AGREE 

AGREE 

NEITHER 
AGREE  NOR 
DISAGREE 

DISAGREE 

STRONGLY 

DISAGREE 

Goal  Setting  - 1. 
Instruction  - 2. 
Feedback  - 3. 

Concrete 


,9. 


Cl  arifying  < 


Reasonable  < 


Relevant 


10. 

11. 

12. 

13. 

'14. 


My  CC  tells  us  what  goals  we  are  supposed  to  reach. 

My  CC  gives  us  instructions  which  are  complete. 

My  CC  gives  us  enough  information  on  how  well  we  are  doing. 

My  CC  gives  very  specific  details  when  explaining  something. 

My  CC  demonstrates  things  to  us  to  help  us  learn. 

My  CC  gets  our  full  attention  when  talking  to  us. 

My  CC  explains  things  to  us  sc  that  we  don't  forget. 

My  CC  lets  us  know  what  things  are  most  important. 

My  CC  encourages  us  to  ask  questions  about  things  we  don't 
understand. 

My  CC  asks  us  questions  to  see  if  we  understand  things. 

My  CC  doesn't  expect  too  much  from  the  training  unit. 

My  CC  Is  not  too  easy  on  us. 

My  CC  gives  us  good  reasons  for  the  things  we  do. 

My  CC  tells  us  how  the  skills  we  learn  at  RTC  are  going  to 
make  us  better  sailors. 

My  CC  treats  us  1 ike  human  beings. 

My  CC  lets  us  know  that  he  has  confidence  in  us. 

My  CC  is  patient  with  us. 

My  CC  calls  us  by  our  own  names. 

My  CC  admits  it  when  he  doesn't  know  something. 

My  CC  tells  us  about  his  experiences  In  the  Navy, 

My  CC  has  a good  sense  of  humor. 


Figure  A-l.  Recruit  Survey  (Cont'd) 
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H 

My  CC  takes  part  of  the  blame  when  the  training  unit  does 
poorly  on  something. 

My  CC  really  cares  about  how  the  training  unit  does. 

My  CC  gives  the  right  amount  of  rewards  and  praise  to 
recruits  when  they  deserve  it. 

My  CC  gives  the  right  amount  of  discipline  to  recruits 
when  they  deserve  it. 


26.  My  CC  is  one  of  the  best  at  RTC. 

Respect  for<(_ 

CC  x27.  I have  great  respect  for  my  CC. 


28. 

29. 


Counsel  ingiT' 

^30. 


31. 


My  CC  can  be  informal  with  recruits. 

My  CC  makes  every  effort  to  help  recruits  who  are  having 
personal  or  motivational  problems. 

My  CC  really  listens  to  recruits  in  order  to  understand 
their  problems  better. 

My  CC  tells  recruits  what  they  can  do  or  where  they  can 
go  to  get  help  with  their  problems. 

My  CC  looks  out  for  the  welfare  of  the  recruits. 

My  CC  does  not  harass  or  belittle  recruits. 

My  CC  treats  recruits  in  a consistent  manner. 

My  CC  is  firm  in  enforcing  rules  and  regulations. 

My  CC  shows  a sincere  concern  for  each  recruit  in  the 
training  unit. 

My  CC  does  not  play  favorites  among  recruits. 


Instructing  - 38.  My  CC  sees  to  it  that  individual  Instruction  is  given  to 
recruits  who  need  it. 


Motivating  - 39.  The  RPO's  in  my  training  unit  do  their  jobs  well. 


Figure  A-l.  Recruit  Survey  (Cont'd) 
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Gen.  Morale  - 40.  My  morale  is  very  high. 


Liking  for  - 41.  So  far,  I like  boot  camp  a lot. 

Boot  Camp 

Esprit  de  - 42.  I have  great  pride  in  being  in  the  Navy, 
corps 

Intention  - 43.  I expect  that  I'll  probably  re-enlist  when  my  first  hitch 
to  re-enlist  is  up. 

Expected  - 44.  After  boot  camp,  I expect  that  I will  like  the  Navy. 

Liking  of  the 

Navy  45.  I am  encouraged  about  my  chances  of  getting  ahead  in  the 

Navy. 

Value  of  - 46.  I think  that  the  training  I am  receiving  at  RTC  is  valuable. 
Training 

Expected  - 47.  I am  enthusiastic  about  the  training  I will  get  after  RTC. 
Liking  of 
the  Navy 

Figure  A-l.  Recruit  Survey  (Cont'd) 
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DIVISION  OFFICER  EVALUATION 

The  following  form  contains  statements  about  the  Company  Commanders  in  your 
Division.  Answer  each  statement  by  making  a mark  on  the  NAVTRAEQUIPCEN  answer 
card  which  you  have  been  given.  Mark  the  card  as  follows: 

A if  you  strongly  agree  with  the  statement. 

T if  you  agree  with  the  statement. 

T~  if  you  neither  agree  nor  disagree  with  the  statement. 

"TT  if  you  disagree  with  lEe  statement. 

~T~  if  you  strongly  disagree  with  the  statement. 

Example:  If  question  #1  said,  "Florida  is  hot  in  the  summer"  and  you  strongly 
agree  with  this,  you  would  make  a mark  in  the  "A"  column  after 
number  1. 


A B C D E 

/ 

Make  your  marks  with  a black  pencil.  Do  not  put  any  names  on  the  answer 
card,  but  do  fill  in  the  Training  Unit  number  of  the  Company  Comnander  you  are 
rating,  and  the  day  of  training. 

For  the  final  item,  #31,  mark  the  form  as  follows: 

A i f you  thi nk  the  CC  i s outstanding. 

IT”  if  you  think  the  CC  is  superior. 

T~  if  you  think  the  CC  is  good. 

“IT  if  you  think  the  CC  is  averaqe. 

Tif  you  think  the  CC  is  FeTow  average. 


I.  DIMENSIONS: 


A.  BEARING 

B.  RELIABILITY 

C.  ADAPTABILITY 

D.  INITIATIVE 


E.  COOPERATIVENESS 

F.  COUNSELING 

G.  MANAGEMENT 

H.  PROBLEM  SOLVING 


I.  DISCIPLINE 

J.  OVERALL  QUALITY 
OF  T.U. 

k.  OVERALL  EVALUATION 
OF  CC 


Figure  A-2.  Division  Officer  Evaluation 
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a 

B 

Z 

NEITHER 

D 

E 

STRONGLY 

AGREE  NOR 

STRONGLY 

AGREE 

AGREE 

DISAGREE 

DISAGREE 

DISAGREE 

DIMENSION 


C - 1.  Adapts  to  new  policies  and  procedures. 

G - 2.  Knows  who  is  responsible  for  each  job. 

D - 3.  Performs  his/her  job  with  enthusiasm. 

G - 4.  Keeps  recruits  productively  busy. 

E - 5.  Does  not  resent  being  told  what  to  do. 

I - 6.  Division  staff  does  not  have  to  bring  order  to  his/her  Training 

Unit  when  he/she  is  not  present. 

E - 7.  Solicits  assistance  from  staff  members  vhen  needed. 

0-8.  Has  done  well  in  teaching  recruits  the  knowledge  and  skills 
required  to  be  good  sailors. 

D - 9.  Gives  1002;  effort  to  his/her  job. 

F - 10.  Makes  every  effort  to  counsel  Individual  recruits  who  are  having 
motivational  or  personal  problems. 

A - 11.  Reports  for  duty  in  appropriate  physical  and  mental  condition. 

II  - 12.  Makes  efforts  to  solve  problems  rather  than  just  grumbling  about 

them. 

C - 13.  Does  not  become  frustrated  in  demanding  situations. 

H - 14.  Does  not  discourage  easily  when  working  on  problems. 

J - 15.  Has  Instilled  in  recruits  the  proper  attitudes  for  becoming  fleet 
personnel. 

B - 16.  Takes  action  without  letting  things  slip. 

B - 17.  Makes  sure  his/her  Training  Unit  is  punctual. 

I - 18.  Does  not  send  recruits  to  Division  staff  when  he/she  could  have 
handled  them. 

G - 19.  Makes  the  best  use  of  his/her  time. 

Figure  A-2,  Division  Officer  Evaluation  (Cont'd) 
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A 

5 

C 

NEITHER 

u 

E 

STRONGLY 

AGREE  NOR 

STRONGLY 

AGREE 

AGREE 

DISAGREE 

DISAGREE 

DISAGREE 

A - 20.  Wears  uniform  properly  and  is  neatly  groomed. 


G - 21.  Sees  to  it  that  all  forms,  records,  reports  from  his/her  Training 
Unit  are  done  properly  and  on  time. 

G - 22.  Has  done  an  effective  job  of  organizing  his/her  Training  Unit. 

C - 23.  Adjusts  to  changes  in  schedules. 

D - 24.  Takes  appropriate  action  without  having  to  be  told  or  ordered. 

A - 25.  Conducts  himself/herself  in  a respectful  military  fashion, 

B - 26.  Follows  orders  promptly  and  efficiently. 

J - 27.  His/her  training  unit  appears  well  trained  when  you  see  them  on 
the  street. 

H - 28.  Recognizes  the  significance  of  problems. 

I - 29.  Has  not  had  excessive  discipline  problems  in  his/her  Training 
Unit. 

E - 30.  Willingly  helps  other  staff  members. 


Key  for 
Item 

A 

6 : 

Z 

D 

E 

#31 

OUTSTANDING 

SUPERIOR 

GOOD 

AVERAGE 

BELOW  AVERAGE 

K - 31.  Overall,  I would  rate  this  Company  Commander: 


Figure  A-2.  Division  Officer  Evaluation  (Cont'd) 
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COMPANY  COMMANDER  SURVEY 

The  following  survey  is  part  of  a project  which  is  being  conducted  jointly 
by  RTC  and  NAVTRAEQUIPCEN.  The  project  has  been  names  "CETS,"  for  Computer- 
ized Evaluation  and  Training  System.  The  questions  on  this  survey  cover 
several  areas:  First,  there  are  some  personal  background  questions,  then 
there  are  attitude  questions  concerning  CC  school,  RTC,  recruits  in  general, 
your  division,  and  general  job  conditions. 

The  results  of  this  survey  will  be  used  along  with  results  of  a D.  0. 

Survey  and  a recruit  survey  to  identify  and  correct  weaknesses  in  company 
commanders,  CC  training,  or  RTC  In  general. 

The  first  four  questions  are  background  questions.  Answer  these  questions 
by  putting  a mark  in  the  box  on  the  NAVTRAEQUIPCEN  answer  card  which  corre- 
sponds to  your  answer.  For  the  rest  of  the  items,  select  a letter  according 
to  whether  you  agree  or  disagree  with  the  statement. 

A if  you  strongly  agree  with  the  statement. 

~B~  if  you  agree  wi tn  the  statement. 

~tT  if  you  neither  agree  nor  disagree  with  the  statement. 

T if  you  disagree  with  The  statement. 

~T~  if  .you  strongly  disagree  with  the  statement. 

Example:  If  question  #12  said,  "Florida  is  hot  in  the  summer"  and  you  strongly 
agree,  you  would  mark  the  card  in  the  "A"  column  after  #12. 

A B C 0 E 

12.  • n a = a 

In  the  space  at  the  top  of  the  card  marked  "Company  Number,"  write  the 
number  of  your  present  or  most  recent  Training  Unit.  In  the  space  marked  “Day 
of  Training,"  please  write  the  date  that  you  are  doing  this  survey. 


A.  RTC  ATTRIBUTES: 


1.  QUALITY  OF  CC  SCHOOL 

2.  QUALITY  OF  RECRUITS 

3.  QUALITY  OF  DIVISION  SUPPORT 

4.  VALUE  OF  TRAINING 


5.  RESPECT  FOR  RECRUITS 

6.  JOB  CONDITIONS 

7.  ATTITUDE  TOWARD  BEING 
A CC 


Figure  A-3.  Company  Conmander  Survey 
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COMPANY  COMMANDER  BACKGROUND  QUESTIONS 

1.  How  old  are  you?  a)  20-25 

b)  25-30 

c)  30-35 

d)  35-40 

e)  40  + 

2.  How  long  have  you  been  in  the  Navy?  a)  0-5 

b 5-10 

c)  10-15 

d)  15-20 

e)  20  + 

3.  What  is  the  largest  number  of  persons  you  have  supervised  before 
becoming  a CC?  a)  0 

b)  1-5 

c)  5-10 

d)  10-20 

e)  20  + 

4.  Did  you  volunteer  for  CC  duty?  a.  yes  b.  no 

For  the  rest  of  the  questions,  select  a letter  according  to  whether 
you  agree  or  disagree  with  each  statement. 


nn 

ii 

■1HM 

■ 

■pUji 

■ 

5. 

6. 
7. 

Quality  8. 
of  CC 
School  9. 

10. 

11. 

12. 

13. 


CC  school  prepared  mu  well 

CC  school  prepared  me  well 

CC  school  prepared  me  wel  1 

CC  school  prepared  me  well 

CC  school  prepared  me  well 

CC  school  prepared  me  well 

CC  school  prepared  me  well 

CC  school  prepared  me  well 

CC  school  prepared  me  well 

procedures. 


in  the  area  of  maintaining  discipline, 
in  the  area  of  counseling  recruits, 
in  the  area  of  motivating  recruits, 
in  the  area  of  infantry, 

in  the  area  of  barracks, 

in  the  area  of  personnel . 

in  the  area  of  lockers, 

in  the  area  of  academics, 

in  the  area  of  RTC  policies  and 


Figure  A-3.  Company  Conmander  Survey  (Cont'd) 
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A 

B 

1 

NEITHER 

B 

E 

STRONGLY 

AGREE  NOR 

STRONGLY 

AGREE 

AGREE 

DISAGREE 

DISAGREE 

DISAGREE 

14.  I think  that  the  training  recruits  receive  at  RTC  will  be  valuable 
Value  to  than  later  on  in  the  Navy, 

of 

Train-  15.  Because  of  training  recruits,  I have  developed  more  confidence 
ing  and  self-respect. 


16.  Being  a CC  is  a job  that  continually  allows  me  to  learn  something 
worthwhile. 

17.  My  work  as  a CC  is  interesting  enough  to  talk  about  with  people 
not  involved  in  recruit  training. 

Atti- 
tude 18.  I prefer  the  work  of  a CC  compared  to  other  jobs. 

Toward 

Being  19.  I feel  that  my  performance  as  a CC  Is  an  important  and  vital 
a CC  service  to  the  Navy. 

(ATBCC) 

20.  It  is  important  for  me  to  be  an  outstanding  CC. 

21.  I am  proud  to  wear  the  red  aiguilette. 

Respect  22.  I have  a great  respect  for  my  recruits, 
for 

Re-  23.  I have  a lot  of  confidence  in  my  recruits. 

cry±t5_ . 

Quality  24.  My  recruits  are  an  asset  to  RTC  and  the  Navy, 
of 

Re-  25.  Recruits  presently  at  RTC  are  easy  to  teach, 
crufts  _ 

" 26.  I get  help  from  Division  Staff  members  whenever  I need  it. 

Qual ity 

of  27,  It  is  easy  to  talk  with  Division  Staff  members  about  problems  I'm 

Divi-  having  with  my  job. 

sion 

Support  28.  I have  a lot  of  confidence  in  the  Division  Staff, 

29.  The  policies  of  my  present  Division  are  fair. 

30.  My  supervisor  keeps  me  posted  on  how  well  I'm  doing. 

Job  31.  I'm  satisfied  with  my  chances  for  advancement  at  RTC. 

Condi- 
tions 32.  It's  not  who  you  know,  it's  what  you  do  that  gets  you  ahead  at  RTC. 


33.  Most  of  the  policies  and  procedures  at  RTC  roke  good  sense. 


Figure  A-3.  Company  Commander  Survey  (Cont'd) 
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A 

B 

C 

NEITHER 

D 

E 

STRONGLY 

AGREE 

AGREE 

DISAGREE 

Job  34.  Changing  policies  and  procedures  when  you  change  divisions  is  not 
Conditions  a great  problem. 


ATBCC  35.  The  rewards  for  doing  a good  job  at  RTC  are  adequate. 

Quality  35.  I've  never  felt  that  my  superiors  were  out  to  get  me. 
of  Di- 
vision 37.  I do  not  get  conflicting  instructions  on  how  to  perform  my  job 
Support 

38.  Do  you  nave  any  particular  suggestions  concerning  an^  aspect  of 
RTC  or  any  comnents  concerning  the  foregoing  questions?  (Please 
detail  your  suggestions  and  advise  why  you  feel  as  you  do  in  the 
space  provided  below.) 


Figure  A-3.  Company  Commander  Survey  (Cont'd) 


RTC  SYSTEM  DATA 


Company  Commander:  Rank 
Rate 

Training  Unit:  Size  on  formation 

Number  of  recruits  added 

Number  of  re-recycled  recruits 

Number  of  recruits  sent  to  another  T.  U. 

Number  of  recruits  sent  to  a special  T.  U. 

Total  setbacks  at  graduation 

Size  at  graduation 

Average  word  knowl  edge/arithmetic  test  score 

Number  of  street  marks 

Number  of  recruit  proficiency  badges 

MED  final  barracks  average 

MED  final  Infantry  average 

MED  final  locker  average 

NED  final  personnel  average 

Figure  A-4,  RTC  System  Data 
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APPENDIX  B 

EXAMPLES  OF  DATA  OUTPUT 


INDIVIDUAL  DATA 

SUMMARY  VARIABLE:  “RECRUIT  SURVEY" 

“COMPANY  COMMANDER:"  BMC  xxx 


CATEGORY 

T.U.  xx 

DIV.  x 

ALL  E-7's 

ALL  CC's 

T.G.  xx 

1. 

CONCRETE 

3.911 

4.123/  49% 

4.244/  9% 

4.177/  45% 

4,114 

2. 

TIMELY 

4.211 

4.135/  50% 

4.261/  41% 

4.165/  50% 

4.034 

3, 

CLARIFYING 

3.621 

3.706/  50% 

3.834/  23% 

3.808/  46% 

3.716 

4. 

REASONABLE 

2.992 

2.843/  51% 

2.819/  91% 

2.835/  55% 

2.784 

5. 

RELEVANT 

3.790 

3.841/  50% 

3.893/  36% 

3.828/  50% 

3.761 

6. 

CONSIDERATE 

3.280 

3.744/  46% 

3.954/  3% 

3.935/  35% 

4.004 

7. 

HUMAN 

4.066 

3.940/  51% 

4.070/  50% 

4.048/  50% 

4.048 

8. 

GOAL  SETTING 

3.772 

3.914/  49% 

4.074/  12% 

3.995/  45% 

3.827 

9. 

INSTRUCTION 

3.863 

4.110/  49% 

4.287/  8% 

4.156/  44% 

4.051 

10. 

FEEDBACK 

3.184 

3.702/  45% 

3.881/  3% 

3.785/  36% 

3,721 

11. 

COUNSELING 

3.832 

3.775/  50% 

3.828/  50% 

3.853/  50% 

3.851 

12. 

MOTIVATING 

3.349 

3.649/  47% 

3.778/  3% 

3.750/  41% 

3.679 

13. 

INSTRUCTING 

3.538 

3.720/  49% 

3.800/  16% 

3.781/  45% 

3.727 

14. 

REWARD  8 PUN 

3.606 

3.695/  50% 

3.845/  23% 

3.722/  48% 

3.521 

Type  'B 1 to  return  to  the  list  of  CC's  in  your  division,  -CNTRL  P-  to 
print  this  page,  or  type  the  number  of  the  category  to  see  the  items 
which  make  up  these  scores. 

NOTE:  For  reasons  of  data  security,  the  data  shown  are  not  actual,  current 
data;  but  are  from  a developmental  test  of  the  system. 

Figure  8-1.  Individual  Data 


AO 


NAVTRAEQU 1PCEN  IH-300 


Figure  8-2.  Grouped  CC  Data 
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NAVTRAEQUIPCEN  IH-300 
APPENDIX  C 

EXAMPLES  OF  TRAINING  MATERIALS 


BEFORE  WE  GO  ANY  FURTHER 
LET'S  TAKE  A QUICK  REVIEW 
OF  JUST  WHAT  CONCRETE  MEANS! 

"CONCRETE"  MEANS: 

A)  GETTING  INVOLVED  WITH  THE 
RECRUITS 

B)  NOT  GIVING  THE  RECRUITS 
MORE  THAN  THEY  CAN  HANDLE. 

C)  BEING  SPECIFIC  AND  USING 
DEMONSTRATIONS 

> C 


(PRESS  EITHER  "A"  OR  “B"  OR  "C".  THEN  PRESS  -RETURN-) 


BEING  CONCRETE  MEANS  BEING  SPECIFIC  AND  USING 
DEMONSTRATIONS.  AS  A COMPANY  COMMANDER,  IT  IS 
VERY  IMPORTANT  TO  BE  CONCRETE  WHEN  DEALING  WITH 
YOUR  RECRUITS.  IF  YOU  ARE,  YOU  WILL  AVOID  BEING 
MISINTERPRETED  BY  YOUR  RECRUITS. 


WHICH  OF  THE  FOLLOWING  STATEMENTS  IS  MORE  CONCRETE? 


A)  AN  EFFECTIVE  CC  IS 
A GOOO  LEADER  AND 
A PRIDE  TO  HIMSELF 
AND  THE  NAVY. 


B)  AN  EFFECTIVE  CC  SETS 
CLEAR  GOAL0,  GIVES 
DETAILED  INSTRUCTIONS, 
AND  PROVIDES  IMMEDIATE 
FEEDBACK. 

> B 


(PRESS  "A"  OR  "B"  THEN  -RETURN-) 


CORRECT! 

THIS  STATEMENT  IS  MORE  CONCRETE  BECAUSE  IT  DESCRIBES  IN  DETAIL 
SOME  OF  THE  PERFORMANCES  OF  AN  EFFECTIVE  CC. 


PRESS  -RETURN- 
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CONCRETE  IN  GOAL  SETTING 


TO  SET  GOALS  CONCRETELY,  YOU  NEED  TO  BE  VERY  SPECIFIC, 

THIS  MEANS  BEING  DETAILED.  TELL  THE  T.  U.  EXACTLY  WHAT 
IT  IS  YOU  EXPECT  THEM  TO  DO. 

THE  FOLLOWING  STATEMENT  MADE  BY  A CC  IS  AN  EXAMPLE  OF  GOAL 
SETTING. 

IS  THIS  STATEMENT  CONCRETE  IN  TERMS  OF  BEING  SPECIFIC? 

(PRESS  "Y"  OR  “N"  AND  PRESS  -RETURN-) 

> N ...ROGERS,  AS  A STARBOARD  LEADING  P.O.,  YOU  SHOULD  SET  A 
BETTER  EXAMPLE  FOR  THE  MEN. 

OF  COURSE!  THE  CC  DOES  NOT  DESCRIBE  IN  DETAIL  WHAT  A BETTER 
EXAMPLE  IS. 


PRESS  -RETURN- 


CONCRETE  IN  GOAL  SETTING 


TO  SET  (GOALS  CONCRETELY,  YOU  NEED  TO  BE  VERY  SPECIFIC. 

THIS  MEANS  BEING  DETAILEO.  TELL  THE  T.  U.  EXACTLY  WHAT 
IT  IS  YOU  EXPECT  THEM  TO  DO. 

THE  FOLLOWING  STATEMENT  MAOE  BY  A CC  IS  AN  EXAMPLE  OF  GOAL 
SETTING. 

IS  THIS  STATEMENT  CONCRETE  IN  TERMS  OF  BEING  SPECIFIC? 

(PRESS  "V  OR  "N"  AND  PRESS  -RETURN-) 

> Y ...I  EXPECT  TO  SEE  OUR  3.2  IN  BARRACKS  IMPROVED 
NEXT  WEEK! 

EXACTLY!  THE  CC  HAS  DESCRIBED  IN  DETAIL  HOW  THE  TRAINING 
UNIT  SHOULO  DO  ON  THEIR  BARRACKS  INSPECTION. 


PRESS  -RETURN- 
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CONCRETE  IN  GOAL  SETTING 


TO  SET  GOALS  CONCRETELY,  YOU  NEED  TO  BE  VERY  SPECIFIC. 

THIS  MEANS  BEING  DETAILED.  TELL  THE  T.  U.  EXACTLY  WHAT 
IT  IS  YOU  EXPECT  THEM  TO  DO. 

THE  FOLLOWING  STATEMENT  MADE  BY  A CC  IS  AN  EXAMPLE  OF  GOAL 
SETTING. 

IS  THIS  STATEMENT  CONCRETE  IN  TERMS  OF  BEING  SPECIFIC? 

(PRESS  "Y"  OR  "N"  AND  PRESS  -RETURN-) 

> Y ...JOHNSON,  TRY  TO  BRING  YOUR  ACADEMICS  SCORE  UP  TO  THE  T.U. 
AVERAGE  THIS  WEEK. 


CORRECT! 

JOHNSON  KNOWS  EXACTLY  WHAT  THE  CC  WANTS. 


PRESS  -RETURN- 


CONCRETE  IN  GOAL  SETTING 


TO  SET  GOALS  CONCRETELY,  YOU  NEED  TO  BE  VERY  SPECIFIC. 

THIS  MEANS  BEING  DETAILED.  TELL  THE  T.U.  EXACTLY  WHAT 
IT  IS  YOU  EXPECT  THEM  TO  DO. 

THE  FOLLOWING  STATEMENT  MADE  BY  A CC  IS  AN  EXAMPLE  OF  GOAL 
SETTING. 

IS  THIS  STATEMENT  CONCRETE  IN  TERMS  OF  BEING  SPECIFIC? 
(PRESS  “Y"  OR  "N“  AND  PRESS  -RETURN-) 

> N ...I  WANT  TO  SEE  SOME  MORE  TEAMWRK  AROUND  HERE! 

CORRECT! 

THE  CC  DOES  NOT  EXPLAIN  IN  DETAIL  WHAT  TEAMWORK  MEANS. 


PRESS  -RETURN- 
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CONCRETE  IN  GOAL  SETTING 

TO  SET  GOALS  CONCRETELY,  YOU  NEED  TO  BE  VERY  SPECIFIC. 

THIS  MEANS  BEING  DETAILED.  TELL  THE  T.  U.  EXACTLY  WHAT 
IT  IS  YOU  EXPECT  THEM  TO  DO. 

THE  FOLLOWING  STATEMENT  MADE  BY  A CC  IS  AN  EXAMPLE  OF  GOAL 
SETTING. 

IS  THIS  STATEMENT  CONCRETE  IN  TERMS  OF  BEING  SPECIFIC? 

(PRESS  "Y"  OR  “M*  AND  PRESS  -RETURN-) 

> N ...THERE  SHOULD  BE  NO  HITS  FOR  GEAR  ADRIFT  AT  BARRACKS 
INSPECTION  TOMORROW. 

INCORRECT! 1 1 

BY  SPECIFYING  GEAR  ADRIFT,  THE  CC  IS  POINTING  OUT  A SPECIFIC 
OBJECT  IN  THE  INSPECTION. 


PRESS  -RETURN' 


CONGRATULATIONS!  YOUR  SCORE  WAS 
5 

OUT  OF  6 CORRECT.  LET’S  CONTINUE  LEARNING  ABOUT  CONCRETE. 


PRESS  -RETURN- 
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